One-Level Cache Memory Design for Scalable SMT Architectures

The cache hierarchy design in existing SMT and superscalar processors is optimized for latency, but not for bandwidth. The size of the L1 data cache did not scale over the past decade. Instead, larger unified L2 and L3 caches were introduced. This cache hierarchy has a high overhead due to the principle of containment, as all the cache blocks in the upper level caches are contained in the lower level cache. It also has a complex design to maintain cache coherence across all levels. Furthermore, this cache hierarchy is not suitable for future large-scale SMT processors, which will demand high bandwidth instruction and data caches with a large number of ports. This paper suggests the elimination of the cache hierarchy and replacing it with one-level caches for instruction and data. Multiple instruction caches can be used in parallel to scale the instruction fetch bandwidth and capacity. A one-level data cache can be split into a number of block-interleaved cache banks to serve multiple memory requests in parallel. An interconnect will be required to connect the data cache ports to the different cache banks. The interconnect will increase the data cache access time. This paper shows that large-scale SMTs can tolerate longer data cache hit times. Increasing the data cache access time from 3 cycles to 5 cycles reduces the IPC by only 2.8%, and increasing it from 3 cycles to 7 cycles will reduce the IPC by 8.9%.